An efficient ensemble pruning algorithm using One-Path and Two-Trips searching approach

Keywords: Ensemble pruning Backtracking algorithm Ensemble Pruning via BackTracking algorithm (EnPBT) One-Path and Two-Trips searching algorithm (OPTT) Pruning efficiency a b s t r a c t Ensemble pruning is substantial for the successful application of an ensemble system. A novel method for Ensemble Pruning via BackTracking algorithm (EnPBT) was proposed by us in our previous work. Back-tracking algorithm can systematically search for the solutions of a problem in a depth-first and jumping manner, suitable for solving all those large-scale combinatorial problems. The validity of EnPBT algorithm has been verified in our previous work. However, the relatively slow pruning speed might be a drawback of EnPBT. Aiming at this problem, an efficient and novel ensemble pruning algorithm is proposed in this paper, i.e. One-Path and Two-Trips (OPTT) ensemble pruning algorithm. It is very fast in pruning speed, while its classification performance has no significant difference with EnPBT, as demonstrated in the experimental results of this work. In short, OPTT achieves a proper trade-off between pruning effectiveness and efficiency. An ensemble refers to a set of component models whose predictions are integrated by simple unweighted voting or weighted voting [1]. Ensemble learning has attracted a lot of attentions from machine learning community since the last decade, owing to its potential in prediction accuracy increasing [2–7]. Remarkable improvement in generalization performance has been observed from ensemble learning in a broad scope of application fields, for example: face recognition [8], optical character recognition [9], scientific image analysis [10,11], medical diagnosis [12,13], financial time series prediction [10], military purposes [14], intrusion detection [15], etc. Despite its remarkable performance, a major drawback of ensemble learning is that, generally, it is necessary to combine a large number of classifiers to ensure that the error converges to its asymptotic value. This entails large memory requirement and slow speed of classification. These aspects can be critical in online applications [16–18]. A possible way to alleviate these shortcomings is the selection of a fraction of the classifiers from the original ensemble. Besides the reduction in complexity, ensemble pruning has other potential benefits. In particular, a subset of complementary classifiers can perform better than the complete ensemble [1,16,17,19–24]. However, it has been proven to be an NP-complete problem to realize ensemble pruning effectively and efficiently. Enumerative algorithm for searching the best subset of member networks is not easily worked for ensemble that contains a large number of constituent models. Greedy …